Robust Attentive Behavior Detection by Non-linear Head Pose Embedding and Estimation

We present a new scheme to robustly detect a type of human attentive behavior, which we call frequent change in focus of attention (FCFA), from video sequences. FCFA behavior can be easily perceived by people as temporal changes of human head pose (normally the pan angle). For recognition of this behavior by computer, we propose an algorithm to estimate the head pan angle in each frame of the sequence within a normal range of the head tilt angles. Developed from the ISOMAP, we learn a non-linear head pose embedding space in 2-D, which is suitable as a feature space for person-independent head pose estimation. These features are used in a mapping system to map the high dimensional head images into the 2-D feature space from which the head pan angle is calculated very simply. The non-linear person-independent mapping system is composed of two parts: 1) Radial Basis Function (RBF) interpolation, and 2) an adaptive local fitting technique. The results show that head orientation can be estimated robustly. Following the head pan angle estimation, an entropy-based classifier is used to characterize the attentive behaviors. The experimental results show that entropy of the head pan angle is a good measure, which is quite distinct for FCFA and focused attention behavior. Thus by setting an experimental threshold on the entropy value we can successfully and robustly detect FCFA behavior. ⋆ This work was done when the author was at IR. 2